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SUMMARY 

In  military  operations  and  emergency  management,  operators  and  commanders  must  rely  on  distributed 
and  automated  systems  for  safe  and  ejfective  mission  accomplishment.  Commanders  and  other  decision 
makers  must  manage  true  real  time  system  properties  at  all  levels;  individual  operators,  stand-alone 
technical  systems,  higher-order  integrated  human-machine  systems  and  joint  operations  forces  alike. 
In  these  high-risk,  dynamical  circumstances,  management  and  utilisation  of  automated  command  support 
can  rapidly  become  an  unmanageable  endeavour.  Coping  with  automation  issues  in  command  and  control 
is  a  challenge  for  practitioners  and  researchers.  New  results,  new  measurement  techniques  and  new 
methodological  advances  facilitate  a  more  accurate  and  deeper  understanding,  generating  new  and 
updated  models.  This  in  turn  generates  theoretical  advances.  This  paper  reports  on  research  from  which 
the  results  led  to  a  breakthrough:  An  integrated  approach  to  information-centred  systems  analysis  to 
support  automation  research  and  development  in  command  and  control. 


INTRODUCTION 

In  military  operations  and  emergency  management,  mission  performance  relies  increasingly  on  distributed 
organizations  and  automated  systems  to  attain  high  safety  and  effectiveness  without  risking  excessive 
resource  depletion.  The  nature  of  such  complex  dynamic  processes  and  operations  are  high-risk  activities, 
where  human  and  artificial  team  players,  widely  distributed  across  the  whole  theatre  of  operations, 
together  perform  tasks  requiring  extreme  mobility,  efficiency,  agility  and  endurance.  The  units  function 
autonomously  for  certain  time  periods  and  in  specific  areas,  but  primarily  they  are  forced  to  co-ordinate 
their  actions  with  one  another.  Management  and  utilisation  of  automated  subsystems  and  command 
functions  will  be  part  of  the  daily  routine. 

Humans  are  required  to  monitor  and  maintain  the  conditions  for  safe  and  effective  operation  of  systems 
like  never  before.  Human  operators  have  to  cope  with  poorly  structured  and  imprecise  knowledge, 
by  employing  a  diversity  of  human  problem  solving  activities.  In  some  cases  the  results  are  interpreted  and 
converted  into  physical  control  signals  to  control  and  influence  some  physical  process.  In  other  cases  the 
results  are  implemented  as  orders,  containing  plans,  tactics  and  procedures  for  other  humans  or  artefacts  in 
other  parts  of  an  organisation  to  follow.  Consequently,  a  number  of  issues  naturally  come  to  mind 
concerning  the  humans  and  the  artefacts  over  which  other  humans  must  exercise  control.  Progress  in 
information  technology  generates  systems  that  enable  a  significant  growth  in  the  amount  of  information 
available  for  judgement  and  decision-making.  Even  the  highest  quality  information  will  generally  be 
associated  with  considerable  uncertainty,  ambiguity,  inaccuracy  and  other  deficiencies.  There  is  a  major 
need  to  link  information  with  experiential  context  in  ways  such  that  it  generates  knowledge  useful  for 
judgement,  scrutiny  and  selection. 
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On  the  battlefield  or  on  the  disaster  site  commanders  and  operators  are  forced  to  routinely  handle  highly 
ambiguous  decision  situations,  leaving  them  with  meagre  capacity  to  handle  critical  demands  on  cognitive 
abilities  and  response  time.  Managing  conditions  like  uncertain  evidence,  ambiguous  information, 
time  pressure,  feedback  delays  and  highly  dynamic  circumstances  is  a  risky  endeavour  for  distributed 
teams  with  limited  information  exchange  abilities.  The  smallest  action  can  trigger  dreadful  consequences. 
Goal  conflicts  frequently  occur  and  must  be  resolved  with  no  external  support.  This  is  a  challenge  for  any 
high-performance  organisation  with  limited  resources.  Part  of  the  problem  is  caused  by  the  fact  that 
technological  advances  have  made  the  retrieval,  production  and  distribution  of  information  so  much  easier 
than  before.  Whereas  information  used  to  be  scarce,  and  having  more  of  it  was  considered  a  good  thing, 
it  seems  that  we  now  have  reached  the  point  of  saturation,  and  need  to  limit  our  use  of  it.  The  commander 
used  to  say  to  his  staff:  -  Give  me  all  the  facts!  The  commander  sometimes  tells  his  staff  today:  -  Turn  off 
those  displays  and  give  me  a  paper  map,  grease  pens  and  some  acetate  sheets! 

Technological  advances  have  reduced  the  natural  selection  processes  that  would  otherwise  have  kept  all 
but  the  most  important  information  from  being  taken  into  account.  There  is  a  significant  increase  in 
clumsy  use  of  technology  (Norman,  1988;  Woods  &  Sarter,  1993;  Sarter  et  al.,  1997),  generating  new 
problems  for  the  users: 

1 .  Instead  of  freeing  up  cognitive  resources,  clumsy  use  of  technology  creates  new  kinds  of  cognitive 
work. 

2.  Instead  of  offloading  tasks,  autonomous,  but  silent  machine  agents  create  the  need  for  team  play, 
co-ordination  and  intent  communication  with  people,  demands  which  are  difficult  for  automated 
systems  to  meet. 

3.  Instead  of  focusing  user  attention,  clumsy  use  of  technology  diverts  attention  away  from  the  job  to 
the  interface. 

4.  Instead  of  aiding  users,  generic  flexibility  creates  new  demands  and  requirements  for  the  human 
operator  to  cope  with. 

5.  Instead  of  reducing  human  erroneous  actions,  clumsy  technology  contains  flaws  that  create  the 
potential  for  unpredictable  erroneous  actions  and  assessments  by  users. 

6.  Instead  of  reducing  knowledge  requirements,  clumsy  technology  demands  new  and  more  intricate 
knowledge  and  skills. 

The  result  is  an  explosion  in  often  irrelevant,  unclear  and  inaccurate  data  fragments,  making  it  ever  more 
difficult  to  see  the  big  picture  (Morin,  Jenvald  &  Worm,  1998a).  In  the  domain  of  dynamic  and  complex 
systems,  it  is  very  important  for  an  operator  to  have  access  to  the  big  picture  of  the  process.  This  helps  the 
operator  to  step  back  and  assess  the  overall  system  status,  and  decide  where  to  look  next  within  the 
system  to  acquire  the  next  piece  of  information  needed.  Effects  of  excessive  information  load  include 
anxiety,  poor  decision-making,  difficulties  in  memorising  and  remembering,  and  reduced  attention  span. 
These  effects  merely  add  to  the  stress  caused  by  the  need  to  constantly  adapt  to  a  changing  situation. 

There  is  a  need  for  a  holistic  view  of  the  advantages  and  shortcomings  of  automated  command  and  control 
support.  Consequently,  a  number  of  research  fields  were  reviewed:  systems  theory,  decision-making  and 
decision  theory,  leadership,  psychophysiology,  information  theory  and  the  science  and  systems  of 
command  and  control. 


THE  ACTION  CONTROL  THEORY  FRAMEWORK 

The  underlying  principle  was  integration  of  well-established  scientific  disciplines  into  a  new  research 
direction.  Action  Control  Theory  (ACT),  a  framework  specifically  composed  to  facilitate  empirically  based 
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conceptual  modelling  of  dynamic,  complex  tactical  systems  and  processes  and  of  their  states  and  state 
transitions.  The  research  areas  constituting  ACT  have  until  now  developed  along  separate  paths  of 
evolution.  However,  now  it  is  time  to  investigate  what  they  might  offer  when  implemented  in  an 
integrated,  cohesive  and  coordinated  manner.  Flach  &  Kuperman  (1998)  concluded  that  it  is  essential  to 
develop  a  unified,  proactive,  CSE-based  approach  in  research  and  systems  design  for  future  warfare 
environments.  We  agree,  and  hold  a  strong  belief  in  the  power  of  integrative  research  approaches  that  are 
built  on  solid  classical  and  innovative  theoretical  work,  using  comprehensive  yet  simple  and  robust 
conceptual  and  specific  models  of  systems,  tasks  and  missions,  supported  by  advanced  experimental  and 
measurement  methods  and  data  analysis  techniques. 

The  resulting  models  can  be  used  for  complex,  multi-level  human-machine  systems  design  in  the  military, 
aviation  and  emergency  response  domains.  Action  Control  Theory  is  a  composite  theoretical  structure, 
derived  from  advances  in: 

1.  Cognitive  Systems  Engineering  (CSE). 

2.  Systems  Theory,  Control  Theory  and  Cybernetics. 

3.  Decision  Making  in  Complex  Command  and  Control. 

4.  Psychophysiology. 

THEORETICAL  CONSTITUENT  1:  COGNITIVE  SYSTEMS  ENGINEERING 

The  area  of  Cognitive  Systems  Engineering  (CSE)  has  grown  at  an  increasing  pace  since  the  first 
significant  contributions  were  published  in  the  1980s  by  Rasmussen  (1983;  1986),  who  introduced  the 
concept  of  skill-based,  rule-based  and  knowledge-based  behaviour  for  modelling  different  levels  of  human 
performance.  Hollnagel  &  Woods  (1983)  made  a  significant  contribution  to  this  field  by  their  definition 
of  a  Cognitive  System  (CS)  as  a  Man-Machine  System  (MMS)  whose  behaviour  is  goal-oriented,  based  on 
symbol  manipulation  and  uses  heuristic  knowledge  of  its  surrounding  environment  for  guidance.  A  CS 
operates  using  knowledge  about  itself  and  the  environment  to  plan  and  modify  its  actions  based  on  that 
knowledge.  According  to  Hollnagel  (1999),  the  definition  has  been  revised  over  the  years  in  order  to 
comprise  new  findings  in  human-machine  systems  research  and  to  cover  a  more  comprehensive  and 
fundamental  set  of  system  properties:  what  the  system  achieves,  what  objectives  it  serves  and  what  its 
intentions  are.  The  current  definition  describes  a  CS  as  a  system  that  can  modify  its  pattern  of  behaviour 
on  the  basis  of  past  experience  in  order  to  achieve  specific  anti-entropic  ends.  For  example,  in  Command 
and  Control  (C^)  tasks  in  military  missions  a  multitude  of  sensor  systems,  communication  systems, 
training  programs,  personnel  and  procedures  are  all  elements  of  the  total  operational  system.  Viewing  this 
system  as  a  CS  permits  the  integration  of  all  existing  control  resources;  operators,  commanders, 
technological  facilities,  doctrine,  tactics,  techniques  and  procedures,  organization  and  training  into  a 
coordinated  system  that  safely  and  efficiently  achieves  its  mission.  The  use  of  CSE  to  model,  analyse, 
and  describe  such  systems  performing  hazardous,  real  time,  high-stake  activities  is  a  powerful  approach, 
given  a  sufficient  understanding  by  the  investigator  of  the  interdependencies  and  linkages  between  other 
research  areas  and  the  CSE  field. 

THEORETICAL  CONSTITUENT  2:  SYSTEMS  THEORY,  CONTROL  THEORY 
AND  CYBERNETICS 

From  the  work  of  Ashby  (1956),  Conant  &  Ashby  (1970),  and  many  others  it  is  well  known  that  most 
complex  systems  have  real-time,  dynamic  properties',  the  system  output  at  a  given  time  is  not  only 
dependent  of  the  input  value  at  this  specific  time,  but  also  on  earlier  input  values,  and  that  a  good  regulator 
of  a  system  has  to  implement  a  model  of  the  system  that  is  to  be  controlled.  According  to  Worm,  (2000), 
the  combined  view  of  control  theory  in  technical  as  well  in  behavioural  domains  is  crucial  for  success  in 
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this  research  area.  When  a  function  is  implemented  at  one  level  of  abstraction,  represented  at  a  second 
level  of  abstraction  and  controlled  at  a  third  level  of  abstraction  the  requirement  for  timely  and  complete 
information  varies  accordingly.  On  the  other  hand,  it  is  not  important  whether  an  automated  system  under 
higher-order  supervision  or  a  highly  qualified  operator  carries  out  a  function  or  mission,  the  operators  and 
the  supervisory  controllers  still  need  to  maintain  an  adequate  situation  understanding.  In  most  situations 
the  active  agents  in  a  dynamic  system,  such  as  soldiers/operators  and  their  closest  commander  or  squad 
leader,  operate  in  a  time  scale  of  seconds  to  minutes.  Their  commanders  and  their  command  and  control 
systems  operate  in  time  scales  of  hours  to  days.  The  key  issue  is  to  implement  a  system  theory-based 
framework  to  cope  with  such  dynamic  properties,  and  of  the  environments  such  systems  operate  in. 
The  mathematical  stringency  and  powerful  formalism  of  systems  theory  makes  it  possible  to  describe  and 
treat  systems  as  diverse  as  technical,  organizational,  economic  and  biological  dynamic  systems  in 
basically  the  same  manner:  as  processes,  or  clusters  of  processes,  with  a  built-in  adherent  or  assigned 
control  system. 

According  to  Conant  &  Ashby,  (1970),  Glad  &  Ljung,  (1989)  and  Brehmer,  (1992),  four  fundamental 
requirements  must  be  met,  if  control  theory  should  be  used  successfully  in  analysis  and  synthesis  of 
dynamic  systems: 

1 .  There  must  be  a  goal  {the  goal  condition). 

2.  It  must  be  possible  to  ascertain  the  state  of  the  system  {the  observability  condition). 

3.  It  must  be  possible  to  affect  the  state  of  the  system  {the  action  condition). 

4.  There  must  be  a  model  of  the  system  {the  model  condition). 

The  mission  objectives  and  the  goals  for  the  joint  cognitive  system  construct  one  of  the  four  principal 
components  of  systems  control,  the  goal  condition.  Thus,  the  surrounding  contextual  information  and  the 
experiential  familiarity  of  the  information  users  are  most  important.  It  is  the  use  of  information  within  the 
context  of  contingencies  in  the  mission  and  the  environment  that  results  in  the  transformation  from 
information  to  knowledge,  which  is  then  interpreted  in  terms  of  context  and  experience  by  sensing 
situations  and  recognising  patterns.  We  form  an  internal  system  model,  which  is  a  homomorphic  system 
representation  to  fulfil  the  model  condition.  This  model  conceptualises  what  we  are  supposed  to  achieve, 
with  what  resources,  within  what  timeframe,  while  simultaneously  considering  what  we  will  be  challenged 
with  once  the  imminent  situation  is  resolved. 

From  observations  of  the  mission  environment  and  securing  our  own  capabilities  to  act  in  a  way  that  the 
system  state  progresses  in  a  desirable  manner  to  meet  the  objective  (thereby  fulfilling  the  observability  and 
action  conditions),  we  recognise  characteristics  similar  to  previously  endured  situations.  We  also  simplify 
the  problem  by  using  our  previous  experience  and  professional  skills  to  construct  internal  models, 
hypotheses,  or  schemata  to  use  for  feedforward  control  on  a  temporary  basis.  We  employ  simplified 
deductions,  “rules  of  thumb”,  based  on  these  hypotheses  and  execute  appropriate  actions  according  to 
those  deductions.  Outcome  feedback  from  these  actions  aimed  at  influencing  the  mission  environment 
enables  us  to  learn  more  about  the  environment  and  the  nature  of  the  forthcoming  course  of  action  in  the 
mission.  We  rework  our  hypotheses,  reinforcing  appropriate  ones  and  rejecting  poor  ones  in  order  to 
augment  and  refine  our  internal  system  model. 


THEORETICAL  CONSTITUENT  3:  DECISION  MAKING  IN  COMPLEX 
COMMAND  AND  CONTROL 

The  conventional  and  classic  Analytical  Decision  Making  approach,  supported  by  normative  theories, 
reduces  decision  making  to  selecting  an  appropriate  action  from  a  closed,  pre-defined  action  set,  and  to 
resolution  of  conflicts  of  choice.  Hence,  the  analysis  of  decision  tasks  concentrates  on  the  generation  of 
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alternatives  and  the  evaluation  of  these  alternatives  aeeording  to  some  eriterion,  usually  expeeted  value. 
Aeeording  to  Lehto  (1997),  Cohen  et  al.  (1998),  Wiekens  (1992)  and  Kleindorfer  et  al.  (1993),  the  most 
familiar  elassieal  framework  for  deeision  making  eontains  two  main  parts:  Bayesian  probability  theory  for 
drawing  inferenees  about  the  situation  at  hand,  and  Multiattribute  Utility  Theory  for  selecting  an  optimal 
action.  There  is  a  lot  to  be  said  about  analytical,  mono-theoretical  approaches,  especially  when 
investigators  and  researchers  claim  they  have  a  stringent  and  formal  theory  which  “takes  care  of  it  all” 
regarding  the  host  of  requirements  in  need  of  fulfilment  for  the  theory  to  hold  in  a  real-world  decision 
situation. 

Brehmer  ( 1 992)  suggested  the  use  of  control  theory  as  a  framework  for  research  in  Distributed,  Dynamic 
Decision  Making.  Brehmer’ s  research  was  based  on  analysis  of  several  applied  scenarios,  e.g.  military 
decision  making,  operator  tasks  in  industrial  processes,  emergency  management  and  intensive  care 
(Brehmer,  1988;  1992).  The  following  results  were  clarified  in  these  analyses: 

1 .  The  decision-making  was  never  the  primary  task.  It  was  always  directed  towards  some  goal. 

2.  A  series  of  decisions  is  required  to  reach  the  goal. 

3.  The  decisions  are  mutually  dependent. 

4.  The  state  of  the  decision  problem  changes,  both  autonomously  and  as  a  consequence  of  the 
decision-maker’s  actions. 

5.  The  situational  dynamics  require  decisions  to  be  made  in  real  time. 

Naturalistic  Approaches  to  Decision  Making 

Zachary  &  Ryder  (1997)  reviewed  decision-making  research  during  the  last  decades.  They  elaborated  on 
the  major  paradigm  shift  in  decision  theory  from  the  analytic,  normative  decision  making  procedures  of 
von  Neumann  &  Morgenstem  (1947),  Simon  (1955)  and  Newell  and  Simon  (1972)  to  descriptive 
Naturalistic  Decision  Making  (NDM)  procedures,  described  by  Klein  (1989;  1993a;  1993b)  as  well  as  by 
Klein  &  Woods  (1993)  and  Orasanu  &  Connolly  (1993).  NDM  applies  to  many  dynamic,  safety-critical 
and  even  dangerous  areas  of  activity  such  as  tactical  command  and  control  in  military  missions, 
fire  fighting,  emergency  response  and  medical  diagnosis.  The  work  of  Zachary  and  Ryder  relates 
strongly  to  Control  Theory,  Cognitive  Systems  Engineering,  Dynamic,  Distributed  Decision  Making, 
and  Command  and  Control  science,  and  presents  a  broad  approach  to  decision  support  systems 
development  and  design.  The  essentials  of  this  paradigm  are  condensed  below: 

1 .  Human  decision-making  should  be  studied  in  its  natural  context. 

2.  The  underlying  task  and  situation  of  a  problem  is  critical  for  successful  framing. 

3.  Actions  and  decisions  are  highly  interrelated. 

4.  Experts  apply  their  experience  and  knowledge  non- analytically  by  identifying  and  effecting  the 
most  appropriate  action  in  an  intuitive  manner. 

Tactical  Team  Decision  Making 

Tactical  decision-making  teams  in  the  modem  warfare  environment  are  faced  with  situations  characterized 
by  rapidly  unfolding  events,  multiple  plausible  hypotheses,  high  information  ambiguity,  severe  time 
pressure,  and  serious  consequences  for  errors  (Cannon-Bowers  et  al.,  1995).  There  are  also  cases  when 
geographical  separation  or  other  forms  of  distributed  environments  in  which  the  teams  operate  impose 
additional  difficulties  Brehmer  (1991).  To  be  able  to  adapt  to  these  situations,  team  members  must 
co-ordinate  their  actions  so  that  they  can  gather,  process,  integrate,  and  communicate  information  timely 
and  effectively.  The  accurate  diagnosis  of  team  performance  shortfalls  and  the  tailoring  of  subsequent 
training  toward  correcting  these  shortfalls  for  the  team  and  individual  team  members  require  systematic 
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performance  assessment  from  multiple  perspectives.  Unfortunately,  it  was  the  case  in  the  past  that 
operational  systems  either  ignored  performance  measurement  completely,  or  treated  it  in  an  unsystematic 
fashion.  This  was  particularly  true  of  complex  systems  where  it  was  difficult  to  assess  performance  with  a 
single  correct  answer,  or  in  situations  where  there  were  several  individual  decision-makers  forced  to 
interact  as  a  team. 


THEORETICAL  CONSTITUENT  4:  PSYCHOPHYSIOLOGY 

Traditionally,  stress  research  has  been  oriented  toward  studies  involving  the  body’s  reaction  to  stressors 
(a  physiological  perspective)  and  the  cognitive  processes  that  appraise  the  event  or  situation  as  a  stressor 
(a  cognitive  perspective).  However,  current  social  perspectives  of  the  stress  response  have  noted  that 
different  people  experiencing  similar  life  conditions  are  not  necessarily  affected  in  the  same  manner. 
There  is  a  growing  interest  in  the  epidemiology  of  diseases  thought  to  result  from  stress.  It  has  been  noted 
that  the  incidence  of  hypertension,  cardiovascular  ailments,  and  depression  varies  with  such  factors  as 
race,  sex,  marital  status,  and  income.  This  kind  of  socioeconomic  variation  of  disease  indicates  that  the 
stressors  that  presumably  dispose  people  toward  these  illnesses  are  somehow  linked  to  the  conditions 
that  people  confront  through  their  history  of  varying  occupational  and  social  position  and  status  in  the 
society.  The  stress  response  is  a  warning  of  a  homeostatic  imbalance  occurring  (Levine  and  Ursin,  1991). 
This  implies  that  the  concept  of  model  error  from  control  theory  once  again  can  be  applied.  The  stress 
response  is  also  mobilizing  physiological  resources  to  improve  performance,  which  is  regarded  as  a 
positive  and  desirable  warning  response.  The  Cognitive  Activation  Theory  of  Stress  (CATS)  describes  the 
phases  of  the  stress  response  as  an  alarm  occurring  within  a  complex  cognitive  system  with  feedback, 
feedforward  and  control  loops,  no  less  but  no  more  complicated  than  any  other  of  the  body’s  self-regulated 
systems  (Eriksen  et  al.,  1999). 

Within  joint  cognitive  systems  performing  complex,  high-risk  military  and  emergency  response  missions 
there  is  a  fundamental  and  profound  connection  between  human  operator  physiological  stress  response 
and  discrepancies  between  expectancies  and  experiences.  Decision-makers  are  not  free  to  make  decisions 
when  they  feel  ready  to  do  so.  Instead,  the  environment  requires  decisions  and  the  decision-maker, 
ready  or  not,  have  to  make  these  decisions  on  demand.  According  to  Brehmer  (1991)  this  causes  stress  in 
dynamic  decision-making  tasks.  In  order  to  cope  with  this  stress,  decision-makers  have  to  develop 
strategies  for  control  of  the  assigned  dynamic  tasks  and  for  keeping  their  own  workload  at  an  acceptable 
level.  Coping  strategies  of  individuals  are  primarily  social  in  nature.  The  manner  in  which  people  attempt 
to  avoid  or  resolve  stressful  situations,  the  cognitive  strategies  that  they  use  to  reduce  threat,  and  the 
techniques  for  managing  tensions  are  largely  learned  from  the  groups  to  which  they  belong.  Although  the 
coping  strategies  used  by  individuals  often  are  distinct,  coping  dispositions  are  to  a  large  extent  acquired 
from  the  social  environment. 


MODELS:  A  HUMAN-CENTRED  MODELLING  STRATEGY 

The  success  of  automated  Command  and  Control  support  depends  not  only  upon  computational  speed  and 
robustness,  but  also  whether  the  designers  have  adequately  supported  the  cognitive  demands  of  the  users’ 
tasks  and  the  impacts  of  technological  change  to  the  organisational  processes.  As  noted  earlier  in  this 
work,  human-machine  interaction  and  decision-making  performance  in  critical  situations  are  dramatically 
affected  by  the  design  of  the  user-artefact  co-operation  (e.g.,  task  allocation,  information  sharing 
requirements,  etc.)  with  respect  to  the  environmental  characteristics  (e.g.,  complexity,  uncertainty, 
dynamics,  level  of  threat,  etc.)  and  the  response  requirements  (e.g.,  timing  and  precision).  Woods  &  Roth 
(1988)  propose  that  mismatches  in  the  system  design  involving  these  factors  result  in  the  ineffective  use  of 
resources  and,  in  the  worst  cases,  disastrous  system  errors  and  failures.  Woods  &  Roth  cite  several  cases 
where  automation  degraded  rather  than  improved  performance  due  to  user-related  design  failures  such  as  a 
lack  of  support  for  supervisory  control  requirements  and  decision-making  strategies,  and  failures  to 


2-6 


RTO-MP-088 


Integrating  Scientific  Disciplines  for 
Automated  Command  Support  in  High-Risk  Missions 


anticipate  the  organisational  impacts  of  technological  change.  This  section  presents  several  aspects  of 
system  design  that  can  improve  the  overall  effectiveness  of  the  collaboration  between  human  decision¬ 
makers  and  their  technological  system  support. 

We  have  already  discussed  the  importance  of  models  for  systems  control  and  situation  understanding,  and 
there  are  a  number  of  lessons  to  learn  from  its  more  specialised  applications  in  different  areas,  such  as 
military  operations,  emergency  response  operations  and  air  traffic  control.  When  a  function  is 
implemented  at  one  level  of  abstraction,  represented  at  a  second  level  of  abstraction  and  controlled  at  a 
third  level  of  abstraction  the  requirement  for  timely  and  complete  information  varies  accordingly.  On  the 
other  hand,  if  it  is  not  important  whether  an  operator  or  an  automated  system  carries  out  a  function  or 
mission  under  higher-order  supervision,  the  operators  and  the  supervisory  controllers  still  need  to  maintain 
an  adequate  situation  comprehension,  originating  from  a  model  of  the  system. 

We  will  go  a  little  deeper  into  some  of  the  concepts  introduced  in  the  earlier  sections,  and  begin  with  a 
more  formal  definition  of  what  a  model  is. 

A  model  is  a  system  C  ~  [E,  M,  Y]  constituted  by: 

1.  A  modelled  system  or  mission  environment  E  =  f  (E,  E)  with  the  total  set  E  =  {Ci}  of  possible 
states  that  can  occur,  and  possible  actions  or  laws  E:  E  ?  E.  For  example,  E  could  be  the  set  of  key 
presses  of  a  computer  operator  or  the  physical  world.  Then  E  is  the  behaviour  of  the  operator  or 
the  laws  of  nature. 

2.  A  modelling  system  M  =  f  (M,  R)  with  internal  model  states  or  representations  M  =  {mk},  and  a 
modelling  function,  which  is  basically  a  set  of  rules,  R:  M  ?  M.  For  example,  M  could  be  a 
numerical  or  symbolic  data  set,  a  sequence  of  neural  signals  or  a  reconstructed  course  of  events  in 
a  mission.  Then  the  rules  R  are  the  processing  activity  of  a  computer,  the  synaptic  electrochemical 
transmissions  between  a  network  of  neurons,  or  the  decisions,  orders  and  actions  of  the  unit 
commander  and  his  staff. 

3.  A  representation  function  Y:  E  ?  R.  For  example,  Y  =  {yn}  could  be  a  measurement  vector, 
a  perception,  or  an  observation. 

When  the  functions  E,  R,  and  E  commute,  then  we  have  m^  =  R(mk.i)  =  R(Y(ei.i))  =  Y(E(ei.i))  =  Y(ei). 
Under  these  conditions  C  is  a  good  model,  and  the  modelling  system  M  can  sufficiently  predict  the 
behaviour  of  the  mission  environment  E.  Then  C  is  called  a  generator  of  predictions  about  E.  On  the  other 
hand,  there  exists  a  possibility  that  M  is  a  model  itself,  implying  that  C  in  that  case  is  a  meta-model. 
In  that  case  C  does  not  generate  a  prediction  directly.  Instead  it  generates  another  model,  which  in  turn  is 
capable  of  generating  predictions  at  other  levels  of  abstraction,  aggregation  and  complexity.  Hence  the 
notion  of  situation  understanding  can  be  described  as  a  hierarchical  knowledge  structure,  recursively 
generating  predictions  about  the  world  and  itself,  thereby  empowering  the  cybernetic  system  (cognitive 
system,  agent)  to  make  decisions  about  its  actions. 

Mental  Representations 

According  to  Rouse  et  al.  (1992)  and  Rouse  et  al.  (1993)  the  importance  of  accurate  and  comprehensive 
mental  models  cannot  be  overestimated.  A  mental  model  synthesizes  the  steps  of  a  process,  and  organizes 
them  as  a  unit.  Allen  (1997)  described  the  main  evidence  of  people  developing  and  utilizing  mental 
models  as  follows: 

1.  Prediction  of  process  steps  and  future  course  of  events. 

2.  Explanation  of  the  cause  of  an  event. 

3.  Diagnosis  of  the  reasons  for  malfunction. 

4.  Training  of  operators,  maintainers  and  other  users. 
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To  be  able  to  share,  develop  and  discuss  mental  models  together  with  other  people,  conceptual  models, 

i.e.,  models  of  mental  models  are  of  great  help.  Allen  (1997)  described  various  classes  of  conceptual 
models:  metaphors,  surrogates,  mappings,  task-action  grammars,  plans  and  propositional  knowledge. 
The  model  condition,  or  Conant’s  and  Ashby’s  law  of  required  model-regulatory  identity  (Conant  and 
Ashby,  1970),  that  stipulates  that  every  good  regulator  of  a  system  must  be  a  model  of  that  system,  is  of 
course  also  valid  when  mental  models  are  considered.  This  was  commented  by  Hollnagel  (1999) 
who  described  the  importance  of  Ashby’s  law  of  requisite  variety  to  mental  model  development  and 
minimisation.  Hollnagel  advocated  a  cybernetic  or  functional  approach  to  modelling  of  human  operators 
and  of  the  mental  models  of  their  surrounding  environment.  A  mental  model  can  be  seen  as  the  basis  for 
generating  input  to  a  system,  in  order  to  keep  the  variety  of  the  system  within  given  limits. 

Tactical  Joint  Cognitive  Systems 

The  point  of  departure  in  this  ACT-based  systems  modelling  endeavour  was  the  definition  of  a  Tactical 
Joint  Cognitive  System  (TJCS)  as  an  aggregate  of  one  or  several  instances  of  four  principal  sub-system 
classes: 

1.  Technological  Systems,  for  example  vehicles,  intelligence  acquisition  systems,  communication 
systems,  sensor  systems,  life  support  systems,  including  the  system  operators. 

2.  Command  and  Control  Systems,  consisting  of  an  information  exchange  and  command  framework, 
built  up  by  technological  systems  and  directly  involved  decision-makers. 

3.  Support  Systems,  comprising  staff  functions,  logistic  functions,  decision  support  functions, 
organizational  structures,  and  various  kinds  of  service  support. 

4.  Tactical  Teams,  composed  and  defined  according  to  (Salas  et  al.,  1992)  as:  “Two  or  more  people 
who  interact,  dynamically,  interdependently,  and  adaptively  toward  a  common  and  valued 
goal/objective/mission,  who  have  been  assigned  specific  roles  or  functions  to  perform,  and  who 
have  a  limited  life-span  of  membership.” 

Mission  Execution  and  Control  Models 

The  next  step  was  integration  of  these  concepts  into  a  Tactical  Action  COntrol  Model  (TACOM). 
The  principal  components  of  the  TACOM  are  the  Mission  Environment,  the  Tactical  Joint  Cognitive 
System,  the  Situation  Assessment  function,  and  the  Cognitive  Action  Control  function,  derived  primarily 
from  the  work  of  Brehmer  (1988;  1992),  Klein  (1993a;  1993b)  and  Worm  (2000;  2001a;  2002).  From  the 
TACOM  the  Mission  Execution  and  Control  Model  (MECOM)  is  constructed.  The  MECOM  consists  of 
one  or  several  TACOMs  extended  with  control-theoretic  components,  to  handle  system  disturbances, 
model  error,  and  to  allow  an  adaptive  and  balanced  mix  of  feedforward  and  feedback  control.  The  last  step 
in  the  model  formation  process  is  combining  and  aggregating  several  MECOMs  into  unilevel  and 
multilevel  MECOMs,  respectively. 


METHODS:  THE  TRIDENT  PROJECT 

We  have  for  several  years  conducted  methodological  development  in  the  Tactical  Real-time  Interaction 
in  Distributed  EnvironmeNTs  (TRIDENT)  project  to  support  the  modelling  work  (Worm,  2001b). 
The  primary  objective  of  TRIDENT  is  to  develop  a  coherent  and  straightforward  package  of  theoretically 
sound  and  empirically  validated  methods  and  techniques  for  human-machine  systems  analysis  in  the 
setting  of  tactical  mission  scenarios.  The  components  of  TRIDENT  are  summarised  below: 

1.  Using  the  Action  Control  Theory  (ACT)  Framework  for  conceptual  modelling  of  dynamic, 
complex  tactical  systems  and  processes,  of  their  states  and  state  transitions. 
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2.  Identification  of  mission  and  unit  state  variables,  and  of  action  control  and  decision  making 
mechanisms  for  process  regulation. 

3.  Mission  Efficiency  Analysis  (Worm  et  al.,  1998;  Worm,  2001a;  2001b)  of  fully  manned  and 
equipped  units  executing  full-scale  tactical  missions  in  an  authentic  environment. 

4.  Measuring  information  distribution  and  communication  effectiveness. 

5.  Measuring  workload  by  means  of  the  NASA  Task  Load  Index  (Hart  &  Staveland,  1988). 

6.  Assessing  team  member  psychosocial  mood  by  means  of  the  Mood  Adjective  CheckList 
(MACE,  Sjoberg  et  al.,  1979). 

7.  Assessing  situation  awareness  (Endsley,  1995)  as  a  function  of  mission-critical  information 
complexity  (Svensson  et  al.,  1993). 

8.  Measuring  level  and  mode  of  cognitive,  context-dependant  control  of  the  team  members, 
and  identifying  what  decision  strategies  were  utilised  by  the  team  and  team  members. 

9.  Applying  reliability  and  error  analysis  methods  for  investigating  failure  causes  both  in  retrospect 
and  for  prediction  (Hollnagel,  1998). 

10.  Validating  identified  constructs  and  measuring  their  influence  using  advanced  data  analytic 
procedures. 

STUDIES 

Numerous  battle  management  and  emergency  response  studies  have  been  carried  out  in  which  we  used 
every  opportunity  to  test,  refine  and  augment  the  modelling,  measurement,  data  collection  and  analysis 
concepts  of  TRIDENT.  Implementing  these  ideas  for  tactical  mission  analysis  in  potentially  dangerous, 
stressful  and  cognitively  complex  environments  showed  to  be  very  effective. 

Using  the  TRIDENT  concepts  for  analysis  and  evaluation  on  aggregated  system  levels  has  so  far  been 
very  rewarding,  with  high  acceptance  among  the  subjects;  trained  and  skilled  professionals  performing 
their  daily  tasks  in  their  accustomed  work  environment.  However,  we  have  also  experienced  some 
critique.  It  is  occasionally  claimed  that  reliability  and  validity  of  subjective  workload  ratings  are 
insufficient.  For  that  reason  we  considered  incorporating  a  measure  of  workload  and  stress  which  is 
commonly  accepted  in  the  scientific  community.  We  considered  hormonal  response  measures,  inspired  by 
the  results  of  Svensson  et  al.  (1993),  who  studied  workload  and  performance  in  military  aviation,  Zeier, 
(1994)  who  studied  workload  and  stress  reactions  in  air  traffic  controllers,  and  Holmboe  et  al.  (1975), 
who  studied  military  personnel  performing  exhausting  battle  training.  We  designed  a  study  in  order  to 
elucidate  to  what  extent  hormonal  physiological  stress  indications  are  linked  to  the  rating,  observation 
and  data  collection  methods  normally  used  in  TRIDENT  to  assess  workload  and  tactical  performance. 
The  details  of  the  study  are  described  in  Worm  (2000). 


RESULTS 


From  the  studies  a  number  of  particularly  interesting  causes  of  mission  failure  or  poor  performance  could 
be  identified.  The  predominant  error  modes  were: 

1 .  Timing  of  movement  and  of  tactical  unit  engagement. 

2.  Speed  of  movement  or  maneuver,  which  is  especially  important  in  the  initial  phase  of 
engagement. 

3.  Selection  of  wrong  object.  The  environments  of  ground  warfare  or  emergencies  offer  many 
opportunities  for  choosing  wrong  objects,  in  navigation,  in  engagements,  or  in  visual  contact. 
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After  a  retrospective  cognitive  reliability  and  error  analysis  using  the  Cognitive  Reliability  and  Error 
Analysis  Method  (CREAM)  developed  by  Hollnagel  (1998)  we  found  that  mission  failure  or  poor 
performance  in  every  case  could  be  attributed  to: 

1 .  Slow  or  even  collapsed  organizational  response. 

2.  Ambiguous,  missing  or  insufficiently  disseminated,  communicated  and  presented  information. 

3.  Equipment  malfunction,  e.g.  power  failure  or  projectile/missile  impact. 

4.  Personal  factors:  inexperience,  lack  of  team  training  etc. 

Empirical  results  suggest  three  potentially  significant  mechanisms  influencing  how  the  team  is  able  to 
execute  mission  control,  which  consequently  also  influences  mission  efficiency: 

1.  Time-dependant  filtering  functions  like  defense  and  coping  mechanisms  according  to  the 
cognitive  Activation  Theory  of  Stress  (Eriksen  et  al.;  1999,  Eevine  &  Ursin,  1991). 

2.  Performance  limiting  factors  due  to  specific  mission  and  task  situation  factors  and  resource 
requirements  (Reason,  1997;  Hollnagel,  1998;  Worm,  2001a). 

3.  Balance  between  feedforward  and  feedback  in  mission-critical  action  control  (Reason,  1997; 
Worm,  2000). 

CONCLUSION 

We  have  for  a  number  of  years  struggled  towards  building  a  foundation  for  analysis  and  evaluation  of 
high-stake,  life-threatening  tactical  missions  in  various  work  contexts.  Although  earlier  results  indicate 
that  a  workable,  reliable  and  valid  result  has  been  achieved,  the  question  is  still  if  the  findings  are 
generally  applicable.  The  theoretical  achievements  were  a  complicated  and  arduous  venture,  in  that  we 
have  constantly  striven  for  empirical  evidence.  Nevertheless  it  is  obvious  that  a  scientific  breakthrough  has 
been  achieved.  We  argue  that  the  ACT  /  TRIDENT  approach  can  be  used  as  an  advanced  systems 
engineering  support  and  will  facilitate: 

1 .  Identification  of  limiting  factors  of  a  specific  individual,  unit,  system,  procedure  or  mission. 

2.  Assessment  of  the  magnitude  of  influence  of  these  factors  on  overall  tactical  performance. 

3.  Generation  and  implementation  of  measures  to  assist,  control  and  improve  insufficient  capabilities 
and  contribute  to  successful  accomplishment  of  future  missions. 

4.  Methodological  support  in  future  integrated  C3I  systems. 

5.  Improving  training  programs  for  tactical  decision-making  and  resource  management. 

Studying  individuals  is  an  effective,  reliable  and  valid  way  to  probe  the  function  and  efficiency  of  an 
organization,  performing  complex  tasks  in  an  ever-changing  mission  environment.  We  will  continue  to 
work  with  collected  data,  and  use  the  results  from  the  analysed  scenarios  to  tune  and  adjust  the  theory, 
models  and  methods  in  order  to  obtain  a  coherent  and  cohesive  framework  for  human-machine  systems 
analysis  of  tactical  mission  settings  and  scenarios.  We  will  also  develop  computerized  versions  of  the  test 
instruments,  if  possible  with  built-in  tools  for  data  analysis  and  graphical  presentation,  so  that  researchers 
and  investigators  not  familiar  with  the  background  and  early  history  of  this  project  can  benefit  in  their  own 
work  from  our  achievements. 

REFERENCES 

Allen,  R.B.,  (1997).  Mental  Models  and  User  models.  In  M.  Helander,  T.K.  Eandauer,  &  P.  Prabhu.  (Eds.) 
Handbook  of  Human-Computer  Interaction,  2nd  Ed.  Elsevier  Science  B.V. 


2  - 10 


RTO-MP-088 


Integrating  Scientific  Disciplines  for 
Automated  Command  Support  in  High-Risk  Missions 


Ashby,  W.R.  (1956).  An  Introduction  to  Cybernetics.  London:  Chapman  &  Hall. 

Brehmer,  B.  (1988).  Organization  of  Decision  Making  in  Complex  Systems.  In  L.P.  Goodstein, 
H.B.  Andersen,  &  S.E.  Olesen  (Eds.),  Tasks,  Errors,  and  Mental  Models.  Eondon:  Taylor  &  Francis. 

Brehmer,  B.  (1991).  Time  Scales,  Distributed  Decision  Making,  and  Modem  Information  Technology. 
In  J.  Rasmussen,  B.  Brehmer,  &  J.  Eeplat  (Eds.),  Distributed  Decision  Making:  Cognitive  Models  for 
Cooperative  Work.  Chichester:  Wiley. 

Brehmer,  B.  (1992).  Dynamic  Decision  Making:  Human  Control  of  Complex  Systems.  Acta  Psychologica, 
81,  pp.  211-241. 

Cannon-Bowers,  J.A.,  Tannenbaum,  S.E,  Salas,  E.,  &  Volpe,  C.E.  (1995).  Defining  Team  Competencies 
and  Establishing  Team  Training  Requirements.  In  R.  Guzzo  &  E.  Salas  (Eds.),  Team  Effectiveness  and 
Decision  Making  in  Organizations,  pp.  333-380.  San  Francisco,  CA:  Jossey  Bass. 

Cohen,  M.S.,  Freeman,  J.T.,  &  Thompson,  B.  (1998).  Critical  Thinking  Skills  in  Tactical  Decision 
Making:  A  Model  and  a  Training  Strategy.  In  J.A.  Cannon-Bowers  &  E.  Salas  (Eds.),  Making  Decisions 
Under  Stress:  Implications  for  Individual  and  Team  Training,  pp.  155-189.  Washington,  DC:  American 
Psychological  Association. 

Conant,  R.C.,  &  Ashby,  W.R.  (1970).  Every  Good  Regulator  of  a  System  must  be  a  Model  of  that  System. 
International  Journal  of  System  Science,  1,  pp.  89-97. 

Endsley,  M.R.  (1995)  Towards  a  Theory  for  Situation  Awareness  in  Dynamic  Systems.  Human  Factors, 
37,  pp.  32-64. 


Eriksen,  H.R.,  Olff,  M.,  Murison,  R.,  &  Ursin,  H.  (1999).  The  Time  Dimension  in  Stress  Responses: 
Relevance  for  Survival  and  Health.  Psychiatry  Research,  85,  pp.  39-50. 

Flach,  J.M.,  &  Kuperman,  G.  (1998).  Victory  by  Design:  War,  Information,  and  Cognitive  Systems 
Engineering.  (Report  No.  AFRE-HE-WP-TR- 1998-0074).  Wright-Patterson  AFB,  OH:  Air  Force 
Research  Eaboratory,  Cre  System  Interface  Division. 

Glad,  T.,  &  Ejung,  E.  (1989).  Reglerteknik.  Grundlaggande  teori.  [Automatic  Control.  Basic  Theory.] 
Fund:  Studentlitteratur. 

Hart,  S.G.,  &  Staveland,  E.E.  (1988).  Development  of  a  Multi-Dimensional  Workload  Rating  Scale: 
Results  of  Empirical  and  Theoretical  Research.  In  P.A.  Hancock  &  N.  Meshkati  (Eds.),  Human  Mental 
Workload.  Amsterdam,  The  Netherlands:  Elsevier  Science  B.V. 

Hollnagel,  E.  (1998).  Cognitive  Reliability  and  Error  Analysis  Method  (CREAM).  Amsterdam, 
The  Netherlands:  Elsevier  Science  B.V. 

Hollnagel,  E.  (1999).  Modeling  the  Controller  of  a  Process.  Trans  Inst  MC,  Vol.  21,  No  4/5, 
pp.  163-170. 

Hollnagel,  E.,  &  Woods,  D.D.,  (1983).  Cognitive  Systems  Engineering:  New  Wine  in  New  Bottles. 
International  Journal  of  Man-Machine  Studies,  18,  pp.  583-600. 

Holmboe,  J.,  Bell,  H.,  &  Norman,  N.  (1975).  Urinary  Excretion  of  Catecholamines  and  Steroids  in 
Military  Cadets  Exposed  to  Prolonged  Stress.  Fdrsvarsmedicin,  1 1,  p.  183. 

Klein,  G.A.  (1989).  Recognition-Primed  Decisions.  In  W.  Rouse  (Ed.),  Advances  in  Man-Machine 
Systems  Research,  5,  pp.  47-92.  Greenwich,  CT:  JAI  Press,  Inc. 


RTO-MP-088 


2  - 11 


Integrating  Scientific  Disciplines  for 

Automated  Command  Support  in  High-Risk  Missions 


ORGAmZATIOM 


Klein,  G.A.  (1993a).  Naturalistic  Decision  Making  -  Implications  for  Design.  State-of-the-Art  Report. 
Dayton,  OH:  Crew  Systems  Ergonomics  Information  Analysis  Center,  Wright-Patterson  Air  Force  Base. 

Klein,  G.A.  (1993b).  A  Recognition-Primed  Decision  (RPD)  Model  of  Rapid  Decision  Making. 
In  G.A.  Klein,  J.  Orasanu,  R.  Calderwood,  &  C.E.  Zsambok  (Eds.),  Decision  Making  in  Action: 
Models  and  Methods.  Norwood,  NJ:  Ablex. 

Klein,  G.A.,  &  Woods,  D.D.  (1993).  Conclusions:  Decision  Making  in  Action.  In  G.A.  Klein,  J.  Orasanu, 
R.  Calderwood,  &  C.E.  Zsambok  (Eds.),  Decision  Making  in  Action:  Models  and  Methods.  Norwood, 
NJ:  Ablex. 

Kleindorfer,  P.R.,  Kunhreuther,  H.C.  &  Schoemaker,  P.J.  (1993).  Decision  Sciences.  An  Integrative 
Perspective.  Cambridge:  Cambridge  University  Press. 

Eehto,  M.R.  (1997).  Decision  Making.  In  G.  Salvendy  (Ed.),  Handbook  of  Human  Factors  and 
Ergonomics,  2nd  Ed.,  pp.  1201-1248.  New  York,  NY:  Wiley. 

Eevine,  S.,  &  Ursin,  H.  (1991).  What  is  Stress?  In  M.R.  Brown,  G-F.  Koob,  &  C.  Rivier  (Eds.), 
Stress  -  Neurobiology  and  Neuroendocrinology.  Marcel  Dekker,  New  York,  pp.  3-21. 

Morin,  M.,  Jenvald,  J.,  &  Worm,  A.  (1998),  Training  Emergency  Management  and  Response:  Creating  the 
Big  Picture.  In  Proceedings  of  The  International  Emergency  Management  Society  1998  Annual 
Conference,  Washington,  DC,  USA. 

Newell,  A.,  &  Simon,  H.A.  (1972).  Human  Problem  Solving.  Englewood  Cliffs,  NJ:  Prentice-Hall. 

Norman,  D.A.  (1988).  The  Psychology  of  Everyday  Things.  New  York:  Basic  Books. 

Orasanu,  J.,  &  Connolly,  T.  (1993).  The  Reinvention  of  Decision  Making.  In  G.A.  Klein,  J.  Orasanu, 
R.  Calderwood,  &  C.E.  Zsambok  (Eds.),  Decision  Making  in  Action:  Models  and  Methods,  pp.  3-20. 
Norwood,  NJ:  Ablex  Publishing  Corporation. 

Rasmussen,  J.  (1983).  Skills,  Rules,  and  Knowledge:  Signals,  Signs,  and  Symbols,  and  Other  Distinctions 
in  Human  Performance  Models.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-13, 
pp.  257-266. 

Rasmussen,  J.  (1986).  Information  Processing  and  Human-Machine  Interaction:  An  Approach  to 
Cognitive  Engineering.  New  York:  North-Holland. 

Reason,  J.  (1997).  Managing  the  Risks  of  Organizational  Accidents.  Ashgate. 

Rouse,  W.B.,  Cannon-Bowers,  J.A.,  &  Salas,  E.  (1992).  The  Role  of  Mental  Models  in  Team  Performance 
in  Complex  Systems.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics,  22,  pp.  1296-1308. 

Rouse,  W.B.,  Edwards,  S.E.,  &  Hammer,  J.M.  (1993).  Modeling  the  Dynamics  of  Mental  Workload  and 
Human  Performance  in  Complex  Systems.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics,  23, 
pp.  1662-1671. 

Salas,  E.,  Dickinson,  T.E.,  Converse,  S.A.,  &  Tannenbaum,  S.I.  (1992).  Toward  an  Understanding  of 
Team  Performance  and  Training.  In  R.W.  Swezey  and  E.  Salas  (Eds.),  Teams:  Their  Training  and 
Performance,  pp.  3-30  Norwood,  NJ:  Ablex  Publishing  Corporation. 


2  - 12 


RTO-MP-088 


Integrating  Scientific  Disciplines  for 
Automated  Command  Support  in  High-Risk  Missions 


Sarter,  N.B.,  Woods,  D.D.,  &  Billings,  C.E.  (1997).  Automation  Surprises.  In  G.  Salvendy  (Ed.), 
Handbook  of  Human  Factors  and  Ergonomics,  2nd  Ed.,  pp.  1926-1943.  New  York:  Wiley. 

Simon,  H.A.  (1955).  A  Behavioral  Model  of  Rational  Choice.  Quarterly  Journal  of  Economics,  69, 
pp.  99-118. 

Sjdberg,  E.,  Svensson,  E.,  &  Persson,  E.-O.  (1979).  The  Measurement  of  Mood.  Scandinavian  Journal  of 
Psychology,  20,  pp.  1-18. 

Svensson,  E.,  Angelborg-Thandertz,  M.,  &  Sjdberg,  E.  (1993).  Mission  Challenge,  Mental  Workload  and 
Performance  in  Military  Aviation.  Aviation,  Space  and  Environmental  Medicine,  Nov.,  pp.  985-991. 

Von  Neumann,  J.,  &  Morgenstem,  O.  (1947).  Theory  of  Games  and  Economic  Behavior.  Princeton, 
NJ:  Princeton  University  Press. 

Wickens,  C.D.  (1992).  Engineering  Psychology  and  Human  Performance.  (2nd  Ed.)  New  York: 
HarperCollins. 

Woods,  D.D.,  &  Roth,  E.M.  (1988).  Cognitive  Engineering:  Human  Problem  Solving  with  Tools.  Human 
Factors,  30,  pp.  415-430. 

Woods,  D.D.,  &  Sarter,  N.B.  (1993).  Evaluating  the  Impact  of  New  Technology  on  Human-Machine 
Cooperation.  In  J.A.  Wise,  V.D.  Hopkin,  &  P.  Stager  (Eds.),  Verification  and  Validation  of  Complex 
Systems:  Human  Factors  Issues.  New  York:  Springer-Verlag. 

Worm,  A.  (2000).  On  Control  and  Interaction  in  Complex  Distributed  Systems  and  Environments. 
Einkoping  Studies  in  Science  and  Technology,  Dissertation  No.  664,  Einkoping  University,  Einkoping, 
Sweden.  ISBN  9 1 -72 1 9-899-0. 

Worm,  A.  (2001a).  Tactical  Mission  Analysis  by  Means  of  Naturalistic  Decision  Making  and  Cognitive 
Systems  Engineering.  In  E.  Salas  &  G.A.  Klein  (Eds.).  Finking  Expertise  and  Naturalistic  Decision 
Making,  pp.  407-431.  Mahwah,  NJ:  Earlbaum.  ISBN:  0-8058-3538-5. 

Worm,  A.  (2001b).  On  Systems  Analysis  and  Performance  Assessment  in  Complex,  High-Risk  Work 
Environments.  Int.  J.  Risk  Assessment  and  Management,  Vol.  2,  Nos.  3/4,  pp.  276-287.  Inderscience, 
Switzerland. 

Worm,  A.  (2002).  An  Integrated  Framework  for  Information-Centered  Human-Machine  Systems 
Analysis.  Int.  J.  Emergency  Management,  Vol.  1,  No.  2,  pp.  125-143.  Inderscience,  Switzerland. 

Worm,  A.,  Jenvald,  J.,  and  Morin,  M.  (1998).  Mission  Efficiency  Analysis:  Evaluating  and  Improving 
Tactical  Mission  Performance  in  High-Risk,  Time-Critical  Operations.  Safety  Science,  30,  pp.  79-98. 
Elsevier  Science  B.V.,  The  Netherlands. 

Zachary,  W.W.,  &  Ryder,  J.M.  (1997).  Decision  Support  Systems:  Integrating  Decision  Aiding  and 
Decision  Training.  In  M.  Helander,  T.K.  Eandauer,  &  P.  Prabhu.  (Eds.)  Handbook  of  Human-Computer 
Interaction,  2nd  Ed.  Elsevier  Science  B.V. 

Zeier,  H.  (1994).  Workload  and  Psychophysiological  Stress  Reactions  in  Air  Traffic  Controllers. 
Ergonomics,  37,  pp.  525-539. 


RTO-MP-088 


2  - 13 


Integrating  Scientific  Disciplines  for 

Automated  Command  Support  in  High-Risk  Missions 


ORGAmZATIOM 


This  page  has  been  deliberately  left  blank 


Page  intentionnellement  blanche 


2  - 14 


RTO-MP-088 


